Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019.

BACKGROUND: Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines. METHODS: We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 and 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U test and Fisher's exact test were performed to compare both groups. RESULTS: Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items. CONCLUSION: The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?

AIM: The aim of this study was to systematically appraise the quality of a sample of COVID-19-related systematic reviews (SRs) and discuss internal validity threats affecting the COVID-19 body of evidence. DESIGN: We conducted a scoping review of the literature. SRs with or without meta-analysis (MA) that evaluated clinical data, outcomes or treatments for patients with COVID-19 were included. MAIN OUTCOME MEASURES: We extracted quality characteristics guided by A Measurement Tool to Assess Systematic Reviews-2 to calculate a qualitative score. Complementary evaluation of the most prominent published limitations affecting the COVID-19 body of evidence was performed. RESULTS: A total of 63 SRs were included. The majority were judged as a critically low methodological quality. Most of the studies were not guided by a pre-established protocol (39, 62%). More than half (39, 62%) failed to address risk of bias when interpreting their results. A comprehensive literature search strategy was reported in most SRs (54, 86%). Appropriate use of statistical methods was evident in nearly all SRs with MAs (39, 95%). Only 16 (33%) studies recognised heterogeneity in the definition of severe COVID-19 as a limitation of the study, and 15 (24%) recognised repeated patient populations as a limitation. CONCLUSION: The methodological and reporting quality of current COVID-19 SR is far from optimal. In addition, most of the current SRs fail to address relevant threats to their internal validity, including repeated patients and heterogeneity in the definition of severe COVID-19. Adherence to proper study design and peer-review practices must remain to mitigate current limitations.

Coronavirus disease (COVID 2019): protocol for a living overview of systematic reviews.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic continues to grow worldwide, and systematic reviews (SRs)/meta-analyses (MAs) on COVID-19 can efficiently guide evidence-based clinical practice. However, SRs/MAs with weaknesses can mislead clinical practice and pose harm to patients, and too many useless SRs/MAs could pose confusion and waste sources. A "living" overview of SRs/MAs aims to provide an open, accessible and frequently updated resource summarizing the highest-level evidence of COVID-19, that can help evidence-users to quickly identify trusted evidence to guide the practice. This study aims to systematically give an overview SRs/MAs of COVID-19, assess their quality, and identify the best synthesis of evidence. METHODS: Databases including Medline, EMBASE, Web of Science, China National Knowledge Infrastructure (CNKI), China Biology Medicine (CBM) and WanFang were systematically searched on May 1, 2020 using relevant terms for identify SRs/MAs related to COVID-19. The study selection, data extraction and quality assessment will be performed by independent reviewers, and results will be crosschecked. The authoritative tools (AMSTAR-2, PRISMA and its extensions) will be used to assess the methodological quality and reporting quality of included SRs/MAs, and potential influence factors will be explored. The consistency of conclusions will be compared among reviews and the best evidence will be summarized. In addition, we will conduct exploratory meta-analyses (MAs) of individual studies when applicable. Data will be reported as number with (or) percentage, risk ratio (RR) or odds ratio (OR), mean difference (MD) or standardized mean difference (SMD) with 95% confidence interval (CI) according to the specific results. R3.6.1 and Microsoft Excel 2016 will be used to analyze and manage data. RESULTS: The results of this overview will be submitted to a peer-reviewed journal for publication. DISCUSSION: In this study, we will present for the first time, an overview of SRs/MAs, which provides a comprehensive, dynamic evidence landscape on prevalence, prevention, diagnosis, treatment, and prognosis of COVID-19.

Assessment of the quality of systematic reviews on COVID-19: A comparative study of previous coronavirus outbreaks.

Several systematic reviews (SRs) have been conducted on the COVID-19 outbreak, which together with the SRs on previous coronavirus outbreaks, form important sources of evidence for clinical decision and policy making. Here, we investigated the methodological quality of SRs on COVID-19, severe acute respiratory syndrome (SARS), and Middle East respiratory syndrome (MERS). Online searches were performed to obtain SRs on COVID-19, SARS, and MERS. The methodological quality of the included SRs was assessed using the AMSTAR-2 tool. Descriptive statistics were used to present the data. In total, of 49 SRs that were finally included in our study, 17, 16, and 16 SRs were specifically on COVID-19, MERS, and SARS, respectively. The growth rate of SRs on COVID-19 was the highest (4.54/month) presently. Of the included SRs, 6, 12, and 31 SRs were of moderate, low, and critically low quality, respectively. SRs on SARS showed the optimum quality among the SRs on the three diseases. Subgroup analyses showed that the SR topic (P < .001), the involvement of a methodologist (P < .001), and funding support (P = .046) were significantly associated with the methodological quality of the SR. According to the adherence scores, adherence to AMSTAR-2 items sequentially decreased in SRs on SARS, MERS, and COVID-19. The methodological quality of most SRs on coronavirus outbreaks is unsatisfactory, and those on COVID-19 have higher risks of poor quality, despite the rapid actions taken to conduct SRs. The quality of SRs should be improved in the future. Readers must exercise caution in accepting and using the results of these SRs.